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THE  SYSTEMS  APPROACH  AND  PUBLIC  POLICY 


E .  S .  Quade 

The  RAND  Corporation,  Santa  Monica,  California 
INTRODUCTION 

The  notion  has  been  around  a  long  time  that  numbers 
and  logic  ought  to  rule  the  universe.  Until  recently, 
however,  only  a  few  philosophers  have  had  much  faith  that 
this  might  actually  come  to  pass.  Quantitative  scientific 
analysis  admittedly  had  a  place  in  engineering,  and  in 
science  itself,  but  for  determining  decisions  and  policy 
in  the  world  of  affairs,  it  had  a. very  limited  value;  that 
world  would  continue  to  be  governed  by  tradition,  judgment, 
intuition,  and  experience.  Wisdom,  insight,  and  persever¬ 
ance  made  Churchill,  Napoleon,  and  Edison  great,  not 
calculation. 

The  systems  approach — and  I'm  encompassing  within  this 
term  systems  analysis,  operations  research,  management 
science,  systems  engineering  and  the  analytic  activ¬ 
ities  associated  with  aerospace  technology — represents  a 
considerable  challenge  to  this  point  of  view.  It  offers 
a  way  to  bring  scientists  and  the  methods  of  science  into 
domains  where  decisions  have  been  almost  the  exclusive 
prerogative  of  politicians,  lawyers,  and  entrepreneurs. 

To  date,,  moreover,  the  systems  approach  has  met  with  fair 


*  I 

Any  views  expressed  in  this  paper  are  those  of  the 
author.  They  should  not  be  interpreted  as  reflecting  the 
views  of  The  RAND  Corporation  or  the  official  opinion  or 
policy  of  any  of  its  governmental  or  private  research 
sponsors.  Papers  are  reproduced  by  The  RAND  Corporation 
as  a  courtesy  to  members  of  its  staff. 
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Institute,  University  of  California,  Los  Angeles. 
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success;  in  commerce/  industry,  and  defense,  systems  '] 

analysis  and  other  scientific  aids  to  decisionmaking  are 
well  accepted — even  extolled — although  not  without  some 

notable  critics.  Elsewhere,  in  attempts  to  deal  with  ' 

problems  of  public  policy,  the  results  are  at  least 
promising  enough  to  generate  considerable  clamor  for  more. 

One  hears  that  to  meet  the  many  challenges  of  our  i 

contemporary  society — the  growth  of  organized  crime,  the 
pollution  of  the  environment,  the  urban  sprawl,  the  world 
food  shortage,  the  international  balance  of  trade,  the 
inadequacies  of  the  U.S.  Post  Office — one  need  only  turn 
to  the  systems  approach. 

However,  in  spite  of  what  we  hear,  it  is  not  clear 
that  the  systems  approach  as  practiced  today — say,  in  the 
aerospace  industry — is  likely  to  be  a  spectacular  success 
with  any  of  these,  even  the  last.  This  is  not  to  say  that 
aerospace  technology,  operations  research,  and  systems 
analysis  cannot  be  successfully  applied  to  specific,  well- 
defined  subproblems  and  subsystems  of  our  state  and  local 
governments.  But  total  problems  are  another  matter.  Even 
the  Post  Office  gives  us  trouble.  We  may  be  able  to  design 
an  efficient  system  but  as  yet  we  have  no  algorithms  for 
finding  ways  to  overcome  the  resistance  offered  by  tra¬ 
dition,  legal  restrictions  and  a  host  of  privileged  interests 
that  inhibit  radical  or  even  morphological  change. 

Nevertheless,  the  outlook  is  hopeful.  My  belief  is 
that  the  systems  approach  can  contribute  significantly  to 
planning  and  policy  formulation  by  federal,  state,  and  local 
governments,  provided  four  developments  now  underway  are 
carried  considerably  farther.  In  two  of  these,  the  analysts 
must  themselves  take  the  lead;  the  other  two  are  largely 

up  to  the  policymakers. 

1  ■  > 

What  the  analysts  must  d©  is  (1)  further  enlarge  their 

concept  of  what  constitutes  acceptable  analysis  and  (2) 
seek  ways  to  adapt  it  to  a  new  decisionmaking  environment. 
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What  public  officials  must -do  is  (3)  tie  the  systems  approach 
into  the  policymaking  system,  giving  it  status  and  respon¬ 
sibility,  and  (4)  provide  the  support  that  is  needed  to  make 
proper  use  of  it. 

I  will  elaborate  on  each  of  these  ideas  but  first  I 
need  to  say  something  about  the  nature  of  systems  approach 
and  how  it  is  currently  practiced. 
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THE  NATURE  OF  THE  SYSTEMS  APPROACH 

What  do  I  mean  by  a  systems  approach?  It  is  not  a 
method  or  technique;  nor  is  it  a  fixed  set  of  techniques; 
but  rather  a  concept,  or  a  way  of  looking  at  a  problem. 

It  is  identical  with  what  I  have  called  systems  analysis 
elsewhere.*  That  is,  it  is  a  practical  philosophy  for 
carrying  out  decision-oriented  interdisciplinary  reseauh, 
a  perspective  on  the  proper  use  of  the  available  tools,  a 
way  to  investigate  how  best  to  aid  a  decisionmaker  faced 
with  complex  problems  of  choice  under  uncertainty.  Opera¬ 
tions  research,  management  science,  systems  engineering, 
and  aerospace  technology  are  either  encompassed  as  techniques 
or  identical,  depending  on  one's  interpretation  of  these 
disciplines. 

The  idea  is  not  new  and,  in  the  abstract,  what  needs  to 
be  done  is  simple  and  rather  obvious.  In  "taking  a  systems 
approach,"  one  must  strive  to  look  at  the  problem  as  a  whole 
and  in  its  proper  context.  Three  sorts  of  inquiry  are 
involved,  any  of  which  can  modify  the  others  as  the  work 
proceeds.  There  is  a  need,  first,  for  a  thorough  investiga¬ 
tion  of  what  the  objectives  of  the  man  or  institution  you 
are  trying  to  help  are  (as  opposed  to  what  he  originally  thinks 
they  may  be)  and  of  the  measures  of  effectiveness  and  criteria 
for  deciding  how  to  choose  among  the  alternatives  that 
promise  to  achieve  those  objectives.  Second,  the  possible 
alternatives  need  to  be  investigated  for  feasibility,  risk, 
and  cost  (in  all  its  aspects),  and  then  compared,  in  terms 
of  the  outcomes  that  follow  from  their  choice.  In  addition, 
if  it  appears  that  we  can  do  significantly  better  or  if  we 
cannot  achieve  the  objectives,  there  must  be  an  attempt  to 

it 

In  reference  (1).  Most  of  the  aerospace  community, 
however,  interprets  systems  analysis  much  more  narrowly, 
restricting  it  to  the  application  of  quantitative  economic 
analysis  and  cost-effectiveness  methods  to  such  matters  as 
vehicle  design  and  the  determination  of  force  composition 
and  deployment. 
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design  further  alternatives  and  select  other  goals.  The 
concept  of  a  systems  approach  is  not  new,  but  what  may  be 
novel  is  the  emphasis:  on  a  long-run  time  horizon;  on 
looking  at  the  entire  problem  as  a  whole?  on  the  clari¬ 
fication  of  objectives;  on  the  search  for  alternatives; 
on  explicitness;  on  the  recognition  of  uncertainty;  and, 
above  all,  on  the  use  of  quantitative  models  to  make 
comparisons  insofar  as  this  can  be  done. 

Systems  analysis,  operations  research,  and  similar 
techniques  have  functioned  well  for  industry  and  defense, 
areas  that  lack  the  benefit  of  a  comprehensive  theory  for 
guiding  action,  by  relying  on  the  systematic  utilization  of 
a  large  body  of  only  partly  articulated  and  largely  intu¬ 
itive  judgment  by  experts  in  the  field.  The  standard 
research  technique  for  such  utilization  is  that  of  con¬ 
structing  an  apprc  riate  model  of  the  situation.  The 
systems  approach  is  now  being  called  on  in  public  policy 
areas  precisely  for  the  same  reasons  it  was  called  on  for 
industry  and  defense — a  theoretical  foundation  for  guiding 
action  is  lacking.  And  again  we  must  rely  on  model  building. 

Without  exception,  insofar  as  I  know,  writers  on  the 
systems  approach  and  related  techniques  recognize  the 
importance  of  model  building  even  though  they  may  not 
always  appreciate  its  full  role.  They  agree  that  the  major 
element  in  tackling  a  problem  is  to  construct  and  use  a  model, 
tailored  both  to  the  situation  under  study  and  to  the  ques¬ 
tion  being  asked.  The  "distinctive  approach"  as  the  Opera¬ 
tional  Research  Society  of  Great  Britain  put  it  in  their 
definition  of  operational  research,  "is  to  develop  a 
scientific  model  of  the  system,  incorporating  measurements 
of  factors  such  as  chance  and  risk,  with  which  to  predict 
and  compare  the  outcomes  of  alternative  decisions,  strategies 
or  controls."  Now  I'm  not  certain  how  the  word  "scientific" 
should  be  interpreted  beyond  a  plea  for  maintaining  the 
same  traditions,  but  I  know  how  it  is  interpreted. 
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The  most  used  models,  often  the  only  type  even  con¬ 
sidered  by  the  usual  practitioneer  are  scientific  in  a  very 
narrow  sense,  that  is,  they  are  mathematical,  expressing 
through  a  set  of  mathematical  equations,  or  a  computer 
program,  the  effectiveness  and  costs  of  alternative  actions 
in  terms  of  variables  which  discriminate  between  the  options 
or  alternatives  and  parameters  which  enable  us  to  investi¬ 
gate  the  contingency  or  context.  By  operating  with  such 
models,  either  analytically  or  numerically,  the  consequences 
of  alternative  choices  are  determined. 

Actually,  systematic  analysis  of  largely  routine 
operations  using  models  of  this  type  is  widespread  through¬ 
out  the  civil  government,  at  all  levels.  For  example,  it 
may  help  to  determine  how  Post  Office  trucks  should  be 
routed  to  collect  mail  from  deposit  boxes,  or  whether  com¬ 
puters  to  handle  inventories  should  be  rented  or  purchased, 
or  how  many  of  each  type  of  approved  textbooks  should  be 
distributed  to  a  given  school.  Such  questions  are  typically 
an  attempt  to  increase  efficiency  in  a  situation  where  it 
is  clear  what  "efficiency”  means.  The  situation  can  be 
modeled,  or,  even  better,  with  minor  modifications,  the 
problem  can  often  be  made  to  fit  a  standard  model  and  the 
analysis  reduced  to  the  application  of  a  well-understood 
mathematical  discipline  such  as  linear  programmi  .g  or 
queueing  theory.  An  "optimum”  solution  is  then  obtained 
by  means  of  a  systematic  computational  routine.  The  queue¬ 
ing  model,  for  example,  may  be  adapted  to  many  operations 
of  freeways,  airports,  service  facilities,  maintenance  shops, 
and  so  on.  In  many  instances,  such  models  may  actually  tell 
the  client  what  his  decision  or  strategy  ought  to  be. 

Models  in  this  tried  but  restricted  sense  are  par¬ 
ticularly  difficult  to  create  when  political  and  social 
factors  predominate.  Sense  examples:  how  much  of  the  city 
budget  should  be  allocated  to  welfare  and  what  portion  of 
that  spent  on  outpatient  clinics,  or  whether  local  transportation 
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needs  are  better  served  by  a  rapid  transit  system  or  by 
more  and  higher  performance  freeways,  or  if  there  is  some 
legislative  action  that  might  end  the  increase  in  juvenile 
delinquency.  Such  questions  involve  more  than  the  alloca¬ 
tion  of  resources.  Here,  rather,  the  objectives  or  goals 
of  the  action  to  be  taken  must  be  determined  first.  It  is 
not  clear  that  "more  efficient"  has  a  meaning  in  these 
problems  and  the  difficulties  almost  always  lie  more  in 
deciding  what  ought  to  be  done  than  in  how  to  do  it. 

In  industrial  and  military  applications  the  problem 
is  far  more  likely  to  deal  with  a  completely  man-made  and 
directed  enterprise — a  manufacturing  process,  a  weapon 
system,  a  railroad  network — something  that  was  designed 
with  a  purpose  in  mind  and  has  a  structure  that  follows 
the  laws  of  engineering  and  economics.  Goals  can  be 
defined,  authority  is  clear-cut  and  cooperative,  and  the 
underlying  design  can  be  discovered  and  modeled.  In  con¬ 
trast,  an  attack  on  problems  of  air  pollution,  urban  renewal, 
vocational  rehabilitation,  or  criminal  justice,  involves 
working  with  goals  that  are  obscure  and  conflicting,  where 
authority  is  diffuse  and  overlapping,  and  where  the  structure 
has  grown  without  conscious  design.  To  discover  the  under¬ 
lying  model  may  require  the  same  sort  of  profound  digging 
that  is  required  to  determine  something  like  the  role  of 
hormones  in  regulating  body  functions.  The  act  of  collection 
may  even  bias  the  data.  It  is  not  surprising,  therefore, 
that  attempts  to  build  quantitative  models  with  which  one 
can  optimise  in  the  conventional  sense  tend  to  fail.  But 
this  does  not  mean  that  we  may  not  be  able  to  find  an 
unconventional  scheme  to  fulfill  really  essential  functions 
of  the  traditional  model.  Unfortunately,  such  schemes  are 
not  always  accepted,  or  at  least  not  much  exploited  in 
standard  practice.  To  change  this  may  require  that  we 
enlarge  our  concept  of  what  a  model  contributes,  if  only 
to  escape  the  view  that  it  is  simply  a  device  to  generate 
values  of  a  "payoff  that  can  be  used  to  determine  preference. 
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The  important  function  of  a  model  in  the  type  of 
analysis  we're  considering  is  to  provide  a  way  to  forecast 
the  outcomes  that  follow  alternative  actions  and,  possibly, 
to  indicate  a  preference  among  them.  A  mathematical 
formulation  with  which  one  can  optimize  is  an  extremely 
valuable  aid  to  this  process  but  it  is  not  crucial;  there 
are  other  routes.  What  is  crucial  to  every  decision  is 
reliance  on  expert  judgment  and  intuition.  This  reliance 
permeates  every  aspect  of  the  systems  approach — in  limiting 
the  extent  of  the  system,  in  deciding  what  hypotheses  are 
likely  to  be  more  fruitful,  in  designing  the  model,  in 
determining  what  the  facts  are,  and  in  interpreting  the 
results.  The  great  virtue  of  model  building  is  that  it 
provides  a  systematic,  explicit,  and  efficient  way  to 
focus  the  required  judgment  and  intuition. 

A  model,  by  introducing  a  precise  framework  and  ter¬ 
minology,  serves  as  an  effective  means  of  communication , 
enabling  analysts  and  various  experts*  to  exercise  their 
judgment  and  intuition  in  a  well-defined  context  and  in 
proper  relation  to  each  other.  In  addition,  it  provides 
feedback  to  guide  the  participants  in  the  revision  of 
their  earlier  judgments.  It  is  these  features  of  the 
model  that  are  essential  to  its  role  in  supplying  a  route 
from  hypotheses  to  prediction,  not  how  explicitly  it 
simulates  the  real  world  or  whether  or  not  it  provides  a 
formal  or  quantitative  scheme  for  optimisation. 

The  realisation  that  this  is  the  case  is  neither  new 
nor  startling.  Operational  gaming,  that  is  to  say,  exercises 
in  which  the  participants  interact  by  playing  roles  that 
simulate  individuals,  or  factions  in  a  society,  or  even 
such  things  as  sectors  in  an  economy,  is  a  first  step 
away  from  the  traditional  model.  Its  predictive  quality 
is  very  clearly  a  function  of  the  intuitive  insight  provided 

k  loose  term,  applied  to  anyone  whose  guidance  and 
knowledge  is  relevant. 
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by  the  participants.  By  allowing  for  the  introduction  of 
judgment  at  every  stage ,  a  game  provides  an  opportunity  to 
take  into  account  intangibles  often  considered  completely 
beyond  the  reach  of  analysis.  Both  the  expert  on  the 
control  team  and  the  player  can  let  their  decisions  be 
influenced  by  their  appraisal  of  the  effects  of  the  simu¬ 
lated  environment.  For  example,  the  player  can  take  into 
account  how  the  success  or  failure  of  an  economic  plan 
may  depend  upon  assumptions  about  a  population's  willing¬ 
ness  to  accept  a  change  in  diet  or  the  flexibility  of  the 
political  structure  to  accommodate  a  new  power  bloc.  In 
any  analytic  formulation  or  computer  simulation,  factors 
of  this  type  must  be  anticipated  and  decisions  about  them 
made  in  advance;  in  a  game  they  can  be  made  seriatim,  and 
in  context,  as  the  need  arises. 

But  gaming — even  thought  it  sacrifices  optimisation — 
still  retains  the  representative  features  of  the  traditional 
model.  My  contention  is  that  there  are  advantages  in  using 
approaches  that  sacrifice  representation  also.  This  brings 
me  to  my  first  point. 
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AN  ENLARGED  CONCEPT  OF  MODEL 

My  suggestion  is  that  we  take  a  broad  view  of  the 
traditional  concept  of  model,  accepting  as  a  substitute 
any  device  that  provides  a  means  to  predict  and  compare 
the  outcomes  of  alternative  actions,  regardless  of  its 
representative  features  or  how  efficient  it  is  at  optimiza¬ 
tion.  Calling  such  a  device  a  "model in  the  context  of 
systems  analysis  and  operations  research  would,  I  think, 
help  to  counter  the  bias  toward  mathematical  models  acquired 
by  so  many  analysts  through  their  education  and  work  with 
industry. 

To  illustrate  that  ther_  are  real  advantages  in  using 
models  of  this  extended  type,  a  discussion  of  one  such 
model,  the  Delphi  procedure,  and  an  outline  of  how  it  might 
be  used  to  tackle  a  problem  with  considerable  social  and 
political  content  should  suffice 

Delphi  is  an  iterative  procedure  for  eliciting  and 
refining  the  opinions  of  a  group  of  people  by  means  of  a 
series  of  questionnaires,  a  "framework"  that  replaces  the 
representative  model.  In  practice,  the  group  would  consist 
of  experts  or  especially  knowledgeable  individuals,  possibly 
including  some  of  the  responsible  decisionmakers  themselves. 
The  idea  is  to  improve  the  usual  panel  or  committee  approach 
in  arriving  at  a  forecast  or  estimate  by  subjecting  the 
views  of  the  individual  participants  to  each  other's  criti¬ 
cism  in  ways  that  avoid  face-to-face  confrontation.  To 
this  end,  the  process  of  deliberation  is  controlled,  through 
feedback,  by  a  steering  group  that  preserves  anonymity  and 
computes  the  group  response  by  using  some  form  of  averaging. 

Anonymity  serves  to  minimize  the  influence  of  vocal 
and  persuasive  individuals  on  group  behavior.  Also  by 
making  all  interactions  between  respondents  go  through 
the  steerinq  group,  "noise" — irrelevant  or  redundant 
material  that  obscures  the  directly  relevant  material 
offered  by  the  participants — can  be  reduced. 
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The  use  of  a  statistical  index,  usually  the  median, 
to  represent  the  group  opinion  is  a  further  device  to 
reduce  group  pressure  toward  conformity.  No  particular 
attempt  to  force  unanimity  among  the  respondents  need  then 
be  made,  and  a  spread  of  opinions  on  the  final  round  is 
the  normal  outcome. 

The  simplest  application  would  be  one  that  requests 
estimates  of  a  set  of  numerical  quantities,  e.g.,  the  dates 
at  which  technological  possibilities  will  be  realized  or 
the  costs  they  will  generate.  The  results  of  a  first 
questionnaire  asking  for  the  quantities  desired  would  be 
summarized  in  terms  of,  say,  the  median  and  inter-quartile 
range  of  the  responses.  This  summary,  possibly  with 
additional  information,  would  then  be  returned  to  the 
original  respondents  with  a  request  for  revision  of  first 
round  estimates  where  appropriate.  On  additional  rounds, 
respondents  would  be  asked  for  further  revisions  and  also 
to  justify  their  estimates.  These  revisions  and  justifi¬ 
cations  are  summarized,  returned,  and  counter  arguments 
requested.  On  further  iterations,  additional  reappraisals 
and  arguments  are  collected.  This  basic  pattern  obviously 
could  have  a  great  many  variants,  only  a  very  few  of  which 
have  been  investigated. 

The  Delphi  procedures — anonymous  response,  iteration, 
controlled  feedback,  numerical  estimates,  statistical 
''group  response" — promise  to  become  a  highly  effective 
means  for  getting  information  out  of  a  group  of  experts. 
Experiments  in  which  the  respondents  seek  answers  to 
"factual"  questions  indicate,  for  example,  that:* 


*  : 

For  the  full  set  of  conclusions  to  these  experiments 
carried  out  at  RAND  in  1968,  see  reference  (2). 


-12- 


1.  Face-to-face  discussion  is  not  as  efficient 
as  more  formalized  communication. 

2.  Improvement  in  accuracy  of  estimates  may 
be  expected  with  iteration. 

3*  Accuracy  improves  when  estimates  of  range 
as  well  as  of  single  points  are  requested 
and  supplied. 

Major  credit  for  development  of  Delphi  must  be  given 
to  Olaf  Helmer  and  Norman  Dalkey  of  RAND.*  My  ideas 
originated  in  their  papers  [3-61.  Although  still  experi¬ 
mental,  Delphi  has  been  used,  among  other  tasks,  to  study 
educational  innovations  [7] ,  to  survey  technological  develop¬ 
ments  of  interest  to  a  commercial  organization  [8] ,  and  to 
provide  short-range  forecasts  of  business  indices  [9] . 

Except  for  the  last,  however,  the  value  of  these  exercises 
is  hard  to  assess  and  further  experimentation  is  urgently 
needed. 

To  illustrate  the  use  of  Delphi  for  systems  analysis, 
let  me  outline  how  it  might  be  applied  to  a  typical  cost- 
effectiveness  problem — allocating  a  budget  for  crime 
prevention . *  * 

On  the  first  round,  one  might  ask  a  panel  drawn  from 
the  policymakers,  their  advisors,  experts  familiar  with  the 
area,  and  possibly  others  who  are  merely  interested,  to 
list  projects  that  they  feel  should  be  included  in  any 
program.  There  will  always  be  alternatives  competing  for 
funds:  more  police,  better  training,  changes  in  court  and 
parole  procedure,  new  laws,  and  so  forth.  Not  all  promis¬ 
ing  projects  can  be  financed;  the  problem  is  to  devise  a 
scheme  to  suggest  and  compare  alternatives,  and  to  select 
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For  an  extended  description  of  another  possible 
application,  see  [1],  pp.  335-342. 
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a  preferred  allocation  of  the  freely  disposable  residue 
of  the  budget. 

A  project  is  rarely,  if  ever,  of  an  all-or-nothing 
kind;  that  is,  there  is  associated  with  it  a  degree  to 
which  it  can  be  executed.  The  problem  therefore  is  not 
merely  to  select  the  best  projects  but  also  to  decide 
how  much  of  each.  In  general,  the  value,  V,  that  will  be 
obtained  from  the  application  of  a  project,  will  typically 
appear  as  a  function  of  its  degree  of  adoption,  q,  in  the 
form  of  an  S-shaped  curve,  as  in  the  figure. 


For  each  project,  the  Delphi  procedure  can  then  be 
used  to  obtain  estimates  from  the  panel  regarding  two  points 
on  this  curve:  the  value  qL  below  which  adoption  of  the 
project  would  be  pointless,  and  the  value  q^  above  which 
the  marginal  benefits  are  so  small  as  to  make  adoption 
wasteful.  (One  would  expect  many  estimates  of  qH  to  be 
zero,  indicating  total  rejection  of  the  project.)  The 
unit  of  measurement  of  q  is  best  in  some  natural  unit, 
such  as  number  of  patrol  cars;  but  if  this  is  not  possible 
some  monetary  unit  may  be  used. 
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After  reaching  the  best  estimates  of  q  and  q  ,  the 
next  step  is  to  work  out  and  estimate  the  cost  of  the  project 
at  each  of  these  values  of  q.  (Of  course  the  expected  cost 
of  a  project  depends  to  some  extent  on  other  projects  that 
are  adopted,  but  we  must  ignore  this  interaction  at  this 
stage.)  We  would  probably  not  need  to  call  on  Delphi  here, 
since  cost  analysis,  like  engineering,  has  a  good  theoretical 
foundation. 

Finally,  comparative  estimates  of  the  effectiveness 
of  each  project  (at  the  proposed  levels  of  adoption)  must 
be  obtained.  This  requires  the  establishment  of  measures 
of  effectiveness,  which  is  not  always  easy  since  the  evalua¬ 
tion  of  some  projects  will  depend  upon  subjective  preferences. 
However,  one  way  to  set  about  this  task  is  as  follows: 
assume  a  scale  of  effectiveness  from  zero  (i.e.,  with  only 
the  precommitted  projects  in  existence)  to  100  (representing 
the  panelists'  assessment  of  the  optimum  effectiveness 
obtainable  from  the  budget) .  Each  panelist  would  then 
give  his  assessment  on  this  scale  of  each  project  implemented 
at  extent  q. 

If  the  estimates  of  the  panel  are  then  combined,  the 
directors  are  now  in  a  position  to  construct,  for  each 
project,  a  curve  of  effectiveness  vs.  cost,  and  the  budget 
can  be  systematically  drawn  so  that  the  associated  values 
of  the  specified  quantities  allocated  form  a  maximum. 

This  approach  has  many  deficiencies.  For  one,  were 
the  budget  to  be  implemented,  it  is  unlikely  that  the 
actual  costs  and  benefits  would  be  identical  with  those 
determined  separately  for  each  project  in  isolation. 
Consequently,  one  or  more  iterations  are  called  for  so  that 
the  participants  may  take  into  account  the  existence  of 
other  projects  at  approximately  the  proper  level.  There 
may  be  more  serious  difficulties  in  this  application, 
however.  The  results  from  any  Delphi  process,  of  course, 
depend  on  two  critical  factors:  the  choice  of  the  panel 
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and  the  way  in  which  the  process  is  implemented.  Here, 
the  choice  of  "policy  advisors"  and  "experts  in  the  area" 
for  the  panel  may  bias  the  outcome  toward  conservatism  in 
dealing  with  a  situation  where  the  only  hope  for  improvement 
may  lie  in  innovation.  And  the  topic  is  shot  through  with 
judgments  about  values  and  goals  that  one  may  not  like  to 
trust  to  such  a  panel,  let  alone  to  the  group  of  analysts 
conducting  the  exercise. 

Much  remains  to  be  learned  about  Delphi  and  the  use 
of  expertise.  For  example,  we  would  like  to  know  how 
much  of  the  convergence  that  takes  place  is  induced  by 
the  process  itself  rather  than  by  elimination  of  the  basic 
causes  of  disagreement.  Placing  the  onus  of  justifying 
their  responses  on  the  respondents  clearly  tends  to  have 
the  effect  of  causing  those  without  strong  convictions 
to  move  their  estimates  closer  to  the  median,  for  those  who 
originally  felt  they  had  a  good  argument  for  a  "deviationist" 
opinion  may  tend  to  give  up.  their  original  estimate  too 
easily;  this  may  result  in  increasing  the  bandwagon  effect 
instead  of  reducing  it  as  intended. 

The  potential  usefulness  of  the  Delphi  approach  is 
much  wider  than  the  published  applications  indicate. 

Extensive  interest  has  been  demonstrated  by  industrial  and 
urban  planners,  research  managers,  policymakers  (in  the 
U.S.  government  and  elsewhere)  in  the  promise  of  Delphi 
procedures  for  technological  forecasting,  corporate  planning, 
organizational  decisionmaking,  and  policy  evaluation. 
Suggested  applications  range  from  the  drafting  of  diplomatic 
notes  to  long-range  political  forecasting.  Unfortunately, 
many  of  the  applications  being  considered  are  marginal  at 
the  moment,  in  the  sense  that  greater  effectiveness  of 
Delphi  procedures  over  more  conventional  techniques  has 
yet  to  be  demonstrated. 

Plans  for  further  investigation  of  the  Delphi  technique 
at  RAND  take  several  directions,  in  addition  to  our  general 
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effort  to  develop  models  for  what  goes  on  and  use  them  to 
improve  the  procedure. 

In  view  of  the  accelerating  use  of  Delphi  procedures 
by  such  a  wide  spectrum  of  public  and  private  institutions, 
two  topics  are  of  immediate  practical  concern:  forecasting 
technological  and  social  events  and  value  judgments. 

With  regard  to  forecasting,  experimentation  in  the 
past  with  short-range  predictions  suggests  that  the  con¬ 
clusions  from  factual  estimation  experiments  apply  to  them 
as  well;  but  this  presumption  needs  confirmation.  Not  much 
can  be  done  experimentally  with  long-range  forecasts  as  far 
as  accuracy  is  concerned,  but  one  may  be  able  to  investigate 
the  reliability  of  such  forecasts ,  in  the  technical  sense 
of  consistency  of  judgments  over  similar  groups  of  "experts." 

In  the  area  of  value  judgments  the  introduction  of 
some  objectivity  can  have  extensive  and  important  repercus¬ 
sions.  As  with  long-range  prediction,  there  is  not  too 
much  that  can  be  done  with  regard  to  "accuracy,"  but  the 
reliability  and  stability  of  group  evaluation  can  be  in¬ 
vestigated  experimentally.  There  is  evidence  from  applied 
exercises  that  iteration  produces  convergence  with  value 
judgments,  but  whether  this  convergence  is  stable  or 
capricious  is  not  visible  from  the  uncontrolled  exercises. 
Finally,  a  large  amount  of  diffuse  "experience"  with  Delphi 
suggests  that  the  structural  properties  of  the  procedure 
(anonymity,  systematic  approach,  controlled  feedback, 

"total"  participation,  statistical  definition  of  group 
response)  leads  to  an  enhanced  acceptance  on  the  part  of 
individual  participants  beyond  what  obtains  with  more 
conventional  (e.g.,  face-to-face)  procedures.  This  is 
clearly  a  valuable  characteristic,  especially  if  the  group 
is  one  of  decisionmakers  or  others  whose  concurrence  is 
required  for  the  implementation  of  policy  and  is  amenable 
to  study. 
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Imperfect  as  it  is,  the  Delphi  process  or  some  further 
modification  appears  to  promise  a  way  to  investigate  many 
problems  with  a  high  social  and  political  content.  Because 
it  can  be  used  to  estimate  the  consequences  of  alternative 
actions,  and  thus  substitute  for  a  conventional  model,  Delphi 
offers  a  hope  of  introducting  the  systems  approach  into  a 
range  of  problems  where  such  models  cannot  be  formulated. 

Now  for  the  second  modification  that  is  needed. 
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ADAPTATION  OF  THE  ANALYSIS  TO  THE  ENVIRONMENT 

In  mili'tary  or  industrial  situations,  systems  analyses 
and  operations  research  studies  are  ordinarily  designed 
and  carried  out,  although  perhaps  not  consciously,  as  if 
they  were  to  assist  an  individual  decisionmaker  (with  the 
authority  to  implement  the  findings)  to  make  a  discrete 
decision.  This  may  be  far  from  a  satisfactory  formulation 
for  the  analysis  of  most  public  policy  problems.  The 
policymaking  environment  is  such  that  there  are  likely  to 
be  many  participants  in  any  given  decision,  most  of  whom 
can  counter  any  recommendation  but  none  with  the  authority 
to  implement  it  alone.  Consequently,  it  seems  desirable 
to  adapt  the  analysis  to  the  changed  nature  of  the  decision¬ 
making  system,  to  pay  specific  attention  to  politics  and 
political  phenomena  and,  in  particular,  to  consider,  as 
part  of  the  analysis,  the  problem  of  how  implementation 
is  to  be  attained. 

In  1966,  for  example,  Edmund  6.  Brown,  then  governor 
of  California,  sought  to  improve  public  administration  there 
by  an  infusion  of  that  new  management  technology,  systems 
analysis.  Contracts  were  let  with  a  number  of  aerospace 
firms  to  investigate  transportation  systems,  criminal 
justice  and  the  prevention  of  delinquency,  the  flow  of 
information  needed  for  the  state's  operation,  the  control 
of  management  of  wastes,  regional  land-use  information 
systems  and  the  state's  social  welfare  operations.  Nothing 
much  has  resulted  from  these  in  the  way  of  action.  But  it 
is  interesting  to  note  that  somewhere  among  their  findings, 
each  study  calls  for  an  increase  in  centralised  authority; 
some  go  so  far  as  to  ask  that  some  sort  of  "environmental" 
manager  or  csar  be  set  up. 

The  pollution  control  and  waste  disposal  study,  for 
instance,  indicated  that  a  profit-making  regional  waste- 
disposal  system  could  handle  all  the  liquid  and  solid 
waste  matter  of  the  entire  San  Francisco  Bay  area,  converting 
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it  to  salable  products,  and  reduce  or  eliminate  the  tax 
burden  required  to  support  this  function.  To  implement 
such  a  system,  the  study  recommended  a  centralized  authority, 
independent  of  city,  county,  or  state  boundaries,  capable 
of  making  decisions  for  the  region  as  a  whole  on  all  matters 
that  might  affect  the  system,  encompassing  finance,  organiza¬ 
tion,  control,  and  the  participants.  In  the  past,  such 
recommendations  have  not  worked  out  very  well  because,  at 
best,  the  central  authority  set  up  has  been  a  temporary 
committee  of  local  representatives  without  true  legal 
decisionmaking  authority. 

In  a  certain  sense,  however,  the  lack  of  authority  to 
implement  is  not  the  only  serious  handicap  facing  a  systems 
approach.  It  is  not  only  very  difficult  to  create  the  kind 
of  inter-governmental  body  required  to  execute  proposed 
systems  solutions  on  a  regional  basis  but  it  is  sometimes 
hard  to  find  an  authority  that  can  thoroughly  review  and 
properly  criticize  an  analysis  that  cuts  across  political 
boundaries  and  suggests  a  regional  solution. 

It  often  seems  to  be  implied  and  sometimes  even  said, 
that  one  purpose  in  introducing  systematic  analysis  into 
the  public  policy  area  is  to  eliminate  politics  from  decision¬ 
making.  If  so,  the  effort  is  doomed  to  failure.  To  a  man 
in  public  office  any  solution  that  extends  past  his  term  is 
long  range,  and  he  must  consider  his  future.  It  is,  in  fact, 
not  only  impractical,  but  unappealing,  to  eliminate  politics 
and  bargaining  in  a  nation  that  guards  its  freedom  from 
centralized  authority.  The  introduction  of  an  analytic 
capability  is  likely  to  change  the  political  power  structure. 
So  much  so  that  the  fear  is  sometimes  expressed  that  a  wide¬ 
spread  dependence  on  analysis  will  lead  to  a  desire  to 
strengthen  central  authority.  What  we  need  is  to  learn  how 
to  do  analysis  that  can  be  used  successfully  in  a  situation 
with  many  decisionmakers  and  diffuse  authority. 


Just  how  to  change  is  not  clear.  The  ideal,  I  suppose 
(although  I  don't  know  how  the  analysis  should  be  carried 
out) ,  is  to  provide  the  responsible  decisionmakers  with  the 
means  to  get  together  and  bring  themselves  to  the  proper 
conclusions.  This  may  mean,  for  example,  advising  on  broad 
policymaking  strategy  rather  than  on  single  discrete  policy 
actions  or  investigating  the  bargaining  positions  and  internal 
needs  of  an  organization. 

In  a  recent  book  [10],  Y.  Dror,  of  Hebrew  University, 
makes  several  suggestions  for  improving  policymaking.  These 
are  based  on  his  contention  that  public  policy  is  made  by  a 
system,  the  public  policymaking  system,  in  which  a  large 
number  of  different  kinds  of  units  interact  in  a  variety 
of  partially  stabilized  but  open-ended  modes.  Action  takes 
place  by  a  process  of  complex  interactions  between  a  large 
number  of  components.  There  are  a  variety  of  paths  leading 
to  any  action,  some  hidden  and  only  partially  known  to  all 
the  participants  and  others  open  and  surrounded  by  a  great 
deal  of  noise.  Proposals  for  changes  must  surpass  many 
hurdles.  One  suggestion  Dror  makes  is  that  each  proposed 
recommendation  be  examined  in  the  analysis  for  political 
feasibility,  that  Is,  the  probability  that  it  will  be  suf¬ 
ficiently  acceptable  to  the  various  secondary  decisionmakers, 
executors,  interest  groups,  and  publics  that  it  can  be  trans¬ 
lated  into  action.  Political  feasibility  depends  on  the 
power  structure  in  which  the  parties  affected  are  involved 
and  on  the  ability  of  the  policymakers  and  the  policy  itself 
to  recruit  upport.  In  addition,  Dror  points  out  how  important 
it  is  that  the  analysis  cover  all  aspects  of  a  policy  even 
if  it  requires  the  use  of  far  less  desirable  means  to  arrive 
at  the  recommendations— for  example,  by  measuring  output 
in  terms  of  input,  say,  by  using  the  professional  qualifica¬ 
tions  of  the  policymakers  to  estimate  the  quality  of  their 
policy. 
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I  am  convinced  that  Dror  is  right  and  that  in  selecting 
an  "optimal"  policy,  political  feasibility  must  be  considered 
in  tha  analysis.  Here  I  am  reminded  of  the  first  item  in 
that  somewhat  remarkable  selection  of  readings  [11] ,  compiled 
as  part  of  their  investigation  of  PPBS  by  the  Senate  Sub¬ 
committee  on  National  Security  and  International  Operations 
under  the  title  Specialists  and  Generalists:  "The  Mice  in 
Council"  from  Aesop's  Fables.  It  illustrates  how  crucially 
important  it  is  that  analysis  consider  feasibility. 

"A  certain  Cat  that  lived  in  a  large  country-house 
was  so  vigilant  and  active,  that  the  Mice,  finding 
their  numbers  grievously  tl..  ned,  held  a  council, 
with  closed  doors,  to  consider  what  they  had  best 
do.  Many  plans  had  been  started  and  dismissed, 
when  a  young  Mouse,  rising  and  catching  the  eye  of 
the  president,  said  that  he  had  a  proposal  to  make, 
that  he  was  sure  must  meet  with  the  approval  of 
all.  "If,"  said  he,  "the  Cat  wore  around  her  neck 
a  little  bell,  every  step  she  took  would  make  i  : 
tinkle?  then,  ever  forewarned  of  her  approach,  we 
should  have  time  to  reach  our  holes.  By  this 
simple  means  we  should  live  in  safety,  and  defy 
her  power."  The  speaker  resumed  his  seat  with 
a  complacent  air,  and  a  murmur  of  applause  arose 
from  the  audience.  An  old  grey  Mouse,  with  s 
merry  twinkle  in  his  eye,  now  got  up,  and  said  that 
the  plan  of  the  last  speaker  was  an  admirable  one? 
but  he  feared  it  had  one  drawback.  Be  had  not 
told  them  who  should  put  the  bell  around  the 
Cat's  neck." 

Now  let  me  turn  to  things  public  officials  must  do 
before  the  systems  approach  can  make  a  really  significant 
impact  on  the  public  policy  area. 
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ANALYSIS  MUST  BE  GIVEN  RECOGNITION 

Scholars  and  citizens  have  been  analyzing  government 
programs  for  centuries,  usually  without  noticeable  results, 
even  when  their  techniques  were  excellent.  But  suggestions 
for  improvement,  whether  they  originate  in  a  letter  to  an 
editor  or  from  a  well  constructed  study  by  an  established 
institution,  to  lead  to  action  must  not  only  reach  the  right 
people  but  surpass  some  attention  threshold.  Even  then,  if 
the  idea  hasn't  reached  some  critical  mass,  it  can  be 
neutralized  by  adjustments  elsewhere  in  the  bureaucracy 
before  it  can  influence  the  system.  To  become  a  really 
effective  part  of  the  policymaking  process,  the  systems 
approach  must  be  established  as  part  of  it  so  that  con¬ 
sideration  of  its  recommendations  is  automatic. 

At  the  Federal  level,  the  introduction,  the  Planning- 
Programming-Budgeting  System  (PPBS) ,  was  designed  to  do  this 

This  system,  ideally  executed  (something  that  may  never 
be  achieved)  would  provide: 

(i)  A  set  of  program  options,  presented  in  a  format 
that  emphasized  the  goals  these  programs  were  designed  to 
achieve.  The  presentation  would  indicate,  for  each  alterna¬ 
tive  over  an  extended  time  period,  what  the  money  would  be 
spent  to  accomplish. . .say,  recreation,  job-training,  fire- 
prevention,  and  so  on— as  opposed  to  how  it  would  be  spent — 
for  things  like  rent,  transportation,  printing,  and  paint. 
The  need  for  choice  would  thus  force  the  decisionmakers  to 
consider  explicitly  whether  or  not  they  are  selecting  the 
best  policy  and  directing  their  resources  to  the  best  use. 

(ii)  An  analytic  process  to  discover  and  design 
alternative  programs,  estimate  their  costs  and  effectiveness 
rank  then  on  various  criteria,  and  supply  arguments  pro  and 
con. 

(iii)  A  data  information  system  to  tell  the  policy¬ 
makers  how  their  programs  are  getting  along  and  to  provide 
material  for  analysis. 
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The  important  characteristics — emphasis  on  objectives,  on 
a  number  of  options,  and  on  an  extended  time  horizon  force 
a  dependence  on  analysis.  This  system  is  in  marked  contrast 
to  conventional  U.S.  government  budgeting  which,  until  the 
introduction  of  the  new  system,  tended  to  present  a  single 
plan,  no  analysis,  a  short  time  horizon,  and  to  emphasize 
what  was  to  be  bought  rather  than  what  was  to  be  done. 

Program  budgeting  has  been  a  long  time  in  winning 
acceptance  in  the  United  States.  Historically,  suggestions 
for  federal  program  budgeting  date  at  least  from  1912  when 
a  commission  reporting  to  President  Taft  proposed  that  a 
budget  bureau  be  established  and  that  government  agencies 
present  their  budgets  along  program  lines.  More  recently, 
two  Hoover  commissions  seeking  to  improve  federal  management 
have  recommended  the  adoption  of  this  type  of  budgeting. 

Also,  even  before  Mr.  McNamara  introduced  it  to  the  Department 
of  Defense,  other  agencies  had  some  form  of  this  type  of 
budgeting . 

The  McNamara  system  thus  was  not  invented  by  Mr. 

McNamara,  If  patent  rights  were  to  be  assigned,  those 
would  belong  to  a  group  of  economists  working  at  The  RAND 
Corporation  in  the  years  1948  to  1960.  In  particular,  the 
ideas  were  first  proposed  in  essentially  the  above  form  by 
Mr.  David  Novick  of  RAND's  Cost  Analysis  Department  and  were 
brought  to  the  Defense  Department  in  1961  by  Charles  Hitch 
when  he  became  Assistant  Secretary  of  Defense,  Comptroller. 
What  was  new  was  the  emphasis  on  analysis  and  the  introduction 
of  concepts  and  procedures  by  which  analysis  could  be  syste¬ 
matically  applied  to  the  processes  of  major  decisionmaking. 

Centering  this  management  system  on  the  budget  is 
another  inspired  perception  of  the  obvious.  The  common 
denominator  of  all  government  decisions  is  that  whatever 
else  they  involve,  they  also  involve  money.  The  budget  is, 
therefore,  the  inevitable  vehicle  within  which  hard  decisions 
involving  competing  claims  for  resources  are  made. 
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President  Johnson  ordered  PPBS  installed  throughout 
the  Federal  government  in  1965.  He  called  the  new  system 
revolutionary,  saying  it  would  make  the  participants  (1) 
define  clearly  the  major  objectives  of  their  programs, 

(2)  apply  systematic  analysis  to  the  alternative  ways  of 
accomplishing  these  objectives,  and  (3)  plan  their  spending 
for  a  long  range  period  as  well  as  for  one  year  ahead. 

Now  this  doesn't  sound  revolutionary,  merely  sensible. 

In  1969  President  Johnson  was  still  enthusiastic, 
crediting  PPBS  with  substantially  improving  the  basis  for 
decisionmaking  within  the  executive  branch.  His  January 
1969  budget  message  recognized  the  impetus  given  to  analysis 
by  the  new  system: 

"The  introduction  to  PPB  has  provided  an  impetus 
toward  increased  use  of  formal  analysis  in  the 
decisionmaking  process.  The  development  and 
consideration  of  alternatives  has  been  stepped 
up,  both  in  the  programming  stage  and  at  the 
budget  decision  stage.  The  emphasis  on  cost 
effectiveness  analysis  as  part  of  the  analytical 
effort  has  drawn  attention  to  ways  of  achieving 
given  objectives  at  least  cost,  or  attaining 
maximum  results  from  given  outlays.  Benefit/ 
cost  analysis,  which  had  been  previously  practiced 
chiefly  in  the  military  agencies  and  the  water 
resources  field,  is  now  underway  on  various 
programs  in  most  major  agencies  of  Government. 

As  experience  has  been  gained,  the  various 
elements  of  the  PPB  approach  and  the  annual 
budget  process  gradually  are  being  more 
effectively  interrelated,  so  that  the  ana¬ 
lytical  results  of  PPB  are  playing  a  greater 
role  in  decisionmaking  for  the  annual  budget. " [12] . 
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Most  observers  would  regard  the  President's  remarks 
as  presenting  far  too  rosy  a  picture.  PPBS  has  not  really 
beer,  tried  on  a  government-wide  basis;  trained  manpower 
to  carry  out  the  analysis  was  lacking  and,  since  in  many 
departments  and  bureaus  its  full  implementation  would 
change  the  power  structure  drastically,  opposition  developed, 
and  the  concept  was  eroded  in  various  ways.  For  instance, 
this  year,  the  requirement  for  specification  of  goals, 
probably  the  most  important  aspect  of  the  whole  idea,  was 
essentially  abandoned. 

Deciding  first  on  the  program  and  the.i  on  the  budget 
rather  than  first  on  the  budget  and  then  on  what  can  be 
done  is  a  useful  idea.  But  merely  to  know,  say,  that  one 
is  planning  to  spend  $100  million  for  salary,  wages,  travel, 
transportation  and  equipment  for  employees  of  the  Office 
of  Education,  does  not  in  itself  provide  information  as 
to  whether  the  results  of  such  activity  are  going  to  be 
worth  the  expenditure.  Here's  where  systems  analysis 
comes  in.  Without  it,  the  planner  can  at  most  have  good 
intentions.  The  decisionmaker  who  seeks  to  spend  $100 
million  for  retarded  children  is  using  $100  million  that, 
through  reduced  taxes,  could  be  used  for  private  expenditure, 
or  for  other  educational  programs,  defense,  mental  health, 
etc.  Analysis  can  help  him  compare  what  the  $100  million 
is  buying  in  each  alternative.  Still  higher  decision¬ 
makers  can  then  use  analysis  to  considf'**  ™hat  that  sum 
would  buy  if  spent  on  other  programs. 

My  last  point  is  that,  if  the  advantages  of  the  systems 
approach  are  really  to  accrue  to  the  public  policymaker, 
there  must  be  a  capability  to  carry  out  the  required  analysis. 
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A  STRONG  ANALYTIC  CAPABILITY  MUST  BE  SUPPORTED 

It  is  universally  agreed  that  methodology  is  not  enough. 
The  analyst,  or  his  team,  must  know,  or  learn  very  quickly, 
a  great  deal  about  the  subject  of  his  analysis.  Public 
policy  problems  are  broad  in  scope;  they  usually  require 
contributions  from  many  disciplines.  A  capable  staff  with 
varied  background  is  needed  to  provide  information,  to 
define  criteria  and  objectives  for  programs,  to  invent  new 
alternatives,  and  also  to  develop  analytic  tools.  In 
addition,  because  effective  analysis  is  not  a  one-shot 
proposition  but  a  dialogue  that  gets  sharper  as  it  goes 
on  and  on  between  the  staff  and  the  policymaker,  the  staff 
must  be  available  for  consultation  and  to  answer  questions. 

Where  should  such  a  capability  be  located  and  how 
supported?  Not  completely  within  and  by  a  single  govern¬ 
mental  entity,  for  the  analysts  must  be  free  from  political 
and  organizational  bias  and  able  to  think  beyond  what  is 
politically  acceptable;  not  completely  in  and  by  industry, 
because  its  knowledge  and  influence  depends  on  privileged 
relations  with  the  government  and  with  competing  industries; 
and  not  solely  in  the  universities,  because  the  studies 
usually  require  an  interdisciplinary  approach  with  close 
cooperation  between  many  skills,  something  difficult  to 
achieve  in  that  environment.  And  it  must  not  be  a  single 
institution,  for  its  own  biases  may  then  dominate.  The 
answer  is  a  mixture;  a  number  of  independent  research 
institutions  maintaining  close  relations  with  government, 
industry,  and  the  universities  but  independent  of  them, 
plus  special  staffs  attached  to  bureaus,  departments,  and 
legislative  bodies. 

In  testimony  before  the  U.  S.  Senate,  Alain  C.  Enthoven, 
then  Assistant  Secretary  of  Defense,  remarked  that  in  their 
experience  "research  funds  in  these  fields  are  likely  to 
be  better  spent  supporting  research  institutions  containing 
groups  of  scholars  from  a  variety  of  relevant  disciplines 
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oriented  toward  the  problems  rather  than  on  individual 
scholars  who  are  more  likely  to  be  oriented  toward  the 
exercise  of  the  academic  specialties"  [13] . 

Even  when  a  major  portion  of  the  studies  is  policy 
oriented,  there  needs  to  be  a  large  program  of  research 
that  explores  technology,  methodology,  and  the  sciences, 
including  the  social  sciences.  Such  studies  are  not 
only  valuable  in  their  own  right,  they  create  a  base  for 
the  policy  studies.  In  order  to  carry  on  such  work,  the 
activity  needs  both  a  broad  charter,  one  that  enables  a 
line  of  research  to  follow  its  findings  to  new  areas  of 
relevance,  and  a  flexible  organization  that  can  bring 
social  and  political  scientists  into  close  cooperation 
with  engineers  and  physical  scientists. 
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IN  CONCLUSION 

There  are  critics  who  see  dangers  in  the  systems 
approach  and  there  are  dangers  where  the  critics  see  them. 
Many  factors  fundamental  to  public  policy  are  not  subject 
to  any  sort  of  quantitative  analysis.  Even  though  systematic 
procedures  exist  (Delphi  is  one  possibility)  for  handling 
factors  that  cannot  be  quantified,  unless  an  effort  is  made 
to  overcome  the  analysts '  bias  toward  quantitative  and 
mathematical  models,  we  may  find  the  more  elusive  political 
and  social  aspects  neglected,  improperly  weighted,  or  even 
deliberately  set  aside.  We  have  not  and  never  may  be  able 
to  make  the  systems  approach  a  pure,  rational,  coldly 
objective,  scientific  aid  to  decisionmaking — only  one  far 
more  so  than  its  alternatives.  But  the  systems  approach 
is  not  a  static  concept — new  procedures  and  techniques  are 
being  constantly  proposed — and  thus  there  is  hope.  If  it 
is  to  be  effective  in  the  area  of  public  policy,  however, 
not  only  must  its  recommendations  be  good#  but  when  they 
are,  it  must  also  be  possible  to  implement  them.  A  good 
analysis  requires  a  capable  staff#  but  implementation  is 
far  more  likely  to  follow  if  the  work  explicitly  considers 
the  problems  of  acceptance  and  is  carried  out  by  a  staff 
that  is,  in  addition#  recognized  and  supported  as  part  of 
the  public  policymaking  system.  Its  recommendations  are  then 
not  easily  neglected  or  shunted  aside. 
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